Virtual care use prior to emergency department admissions during a stable COVID-19 period in Ontario, Canada

Background The increased use of telemedicine to provide virtual outpatient visits during the pandemic has led to concerns about potential increased emergency department (ED) admissions and outpatient service use prior to such admissions. We examined the frequency of virtual visits use prior to ED admissions and characterized the patients with prior virtual visit use and the physicians who provided these outpatient visits. Methods We conducted a retrospective, population-based, cross-sectional analysis using linked health administrative data in Ontario, Canada to identify patients who had an ED admission between July 1 and September 30, 2021 and patients with an ED admissions during the same period in 2019. We grouped patients based on their use of outpatient services in the 7 days prior to admission and reported their sociodemographic characteristics and healthcare utilization. Results There were 1,080,334 ED admissions in 2021 vs. 1,113,230 in 2019. In 2021, 74% of these admissions had no prior outpatient visits (virtual or in-person) within 7 days of admission, compared to 75% in 2019. Only 3% of ED admissions had both virtual and in-person visits in the 7 days prior to ED admission. Patients with prior virtual care use were more likely to be hospitalized than those without any outpatient care (13% vs 7.7.%). Interpretation The net amount of ED admissions and outpatient care prior to admission remained the same over a period of the COVID-19 pandemic when cases were relatively stable. Virtual care seemed to be able to appropriately triage patients to the ED and virtual visits replaced in-person visits ahead of ED admissions, as opposed to being additive.


Introduction
The COVID-19 pandemic has led to the emergence of standard use of virtual visits in health care across the globe [1,2]. In Ontario, Canada the proportion of ambulatory visits completed virtually has been maintained at slightly above 50% from 2020 to 2021 [3]. Despite its widespread adoption, it is still unclear when virtual visits are clinically appropriate and how such wide use of virtual visits impacts patient outcomes and healthcare utilization metrics.
Before the pandemic, there had been concerns that virtual visits may lead to an increased use of outpatient services with patients having both a virtual and an in-person visit for the same clinical issue in situations where the virtual visit was not sufficient [4,5]. For example, pre-pandemic data (2007)(2008)(2009)(2010)(2011)(2012)(2013)(2014)(2015)(2016) from Manitoba showed that virtual visit users had on average 1.3 times more ambulatory visits than non-users [6]. In addition, studies have produced mixed evidence with regard to the effect of virtual visits on urgent services such as emergency department (ED) admissions and hospitalizations [7]. Many of the studies reported in the literature are based on data from site-specific programs and therefore have limited generalizability. Finally, policymakers and some physicians have become concerned that the high rates of virtual visits during COVID-19 have led to an increase in emergency department admissions because of poor access to in-person outpatient care [8]. This concern is exacerbated when one considers rural and lower socioeconomic status patients who already had poor access to care before the pandemic [9]. Combined with reports of lower uptake of virtual visits among these patients [10,11], it is not clear how the transition of care from in-person to virtual impacts ED use.
The high adoption of virtual visits during the pandemic, in the context of a publicly funded healthcare system allowing us access to most visits across the entire population, offers a unique opportunity to examine the frequency of virtual visits use prior to ED admissions. Therefore, the goal of this study was to characterize the frequency and modality (in-person vs virtual) of outpatient care prior to ED admissions. If virtual care replaces in-person care, then we should expect that the proportion of people with both in-person and virtual care before ED admissions should be high relative to those with either modality alone. We, therefore, examined whether there was an overall increase in outpatient visits prior to ED admissions during a period of the pandemic when access to virtual visits was available compared to a seasonality matched period before the pandemic where access to virtual visits was quite limited.
We also aimed to characterize the patients who had a virtual visit prior to an ED admission vs. those who had an in-person visit and the physicians who saw patients with virtual only visits prior to their ED admission compared to those who saw patients virtually or in-person prior to their ED admission.

Study design and population
We conducted a retrospective, population-based, cross-sectional analysis using linked health administrative data in Ontario, Canada to identify all patients who had an ED admission between July 1 and September 30, 2021 and those with ED admissions between July 1 and September 30, 2019. The 2021 summer window represented a relatively stable period of COVID-19, in between the major waves of COVID-19 infection and the 2019 period served as a control period during which access to virtual visits was relatively limited. We excluded patients who had invalid identification numbers, were non-Ontario residents, had ED admissions that were not publicly insured, and those who had another ED admission within 7 days prior to July 1 of the year of interest (full exclusion list is provided in the S1 Appendix).
Data sources. Ontario is the province with the largest population in Canada and all permanent residents have public insurance fully covering all necessary physician and hospital services. We used the Ontario Health Insurance Plan (OHIP) for physician claims, the Canadian Institutes of Health Information Discharge Abstract Database (CIHI-DAD) for information about all hospitalizations, the CIHI National Ambulatory Care Reporting System (NACRS) for hospital-and community-based ambulatory care including ED admissions, and the ICES Physician Database (IPDB) for data on physician specialty. Databases were linked using unique identifiers and analyzed at ICES (formerly the Institute for Clinical Evaluative Sciences). Virtual visits were identified as any OHIP claim with the location recorded as "P" for phone, indicating virtual visit services. We then described patients based on characteristics, such as age, sex, chronic disease diagnoses, income quintile (based on postal code), urban vs. rural (based on postal code's rurality index for Ontario (RIO) where a score below 40 was categorized as urban), Ontario Marginalization Index (ONMARG) containing data on patient economic, ethno-racial, age-based, and social deprivation (details on databases used in the S1 Appendix).
Patient groups. Patients were categorized into subgroups based on the type of outpatient visit (in-person vs virtual) they had prior to their ED admission (if any). For patients who had virtual visits prior to their ED admission (within 7 days), subgroups of patients were also created based on whether their last virtual outpatient visit was within 24, 48, 72 hours, or 7 days of their ED admission. The following non-mutually exclusive groups of patients were created: 1. all patients with an ED admission during the study window 2. patients with an ED admission who did not have any virtual visits within the 7 days prior (includes patients who had zero visits or only in-person visits prior to ED admission) 3. patients with an ED admission who did not have any outpatient visits (virtual or in-person) within the 7 days prior 4. patients who had at least one virtual visit within 24 hours prior to an ED admission 5. patients who had at least one virtual visit within 48 hours prior to an ED admission 6. patients who had at least one virtual visit within 72 hours prior to an ED admission 7. patients who had at least one virtual visit within 7 days prior to an ED admission 8. patients who had only in-person visits during the 7 days prior to admission.
Patients may have belonged to more than one subgroup, i.e. the 48 hours virtual care group includes patients in the 24 hours group. All analyses were conducted using SAS version 9.4.
Patient and physician characteristics. To compare the characteristics of patients who had a virtual vs. an in-person visit prior to ED admission, we looked at the most recent visit that occurred within 7 days prior to their ED admission. We identified patient characteristics such as age, sex, region of residence, rurality, neighborhood income, marginalization index, chronic conditions, number of ED admissions, hospitalizations and outpatient visits in the year prior to ED admission, number of outpatient visits in the 7 days prior to ED admission, days between outpatient visit and emergency visit, whether the visit was on the same day as the ED admission, and if the ED admission resulted in hospitalization.
To examine characteristics of the outpatient visits that occurred within 7 days before ED admissions, we classified patients into 4 groups. Three of these groups included patients who had visited the ED in 2021: those with any visit (virtual or in-person), those with a virtual visit, and those with an in-person visit prior to ED admission. The last group consisted of patients who had an in-person visit prior to ED admission in 2019 (we did not include a virtual visit group as virtual visits were uncommon in 2019).
We also looked at the characteristics of physicians who had provided the most recent outpatient visit prior to the patient's ED admission in 2021 and categorized them into two groups based on the type of visit: physicians who had seen patients with only a virtual visit in 2021; physicians who had seen patients with either a virtual or in-person visit in 2021. For physician characteristics, we looked at age, sex, region of and years in practice, and patient daily volume.
See S1 Appendix for detailed definitions.

Ethics
Use of these databases for the purposes of this study was authorized under §45 of Ontario's Personal Health Information Protection Act, which does not require review by a research ethics board (REB). All data was de-identified at ICES and individual patient consent was waived.

Results and discussion
Between July 1, 2021 and September 30, 2021, there were 1,080,334 ED admissions in Ontario and 74% of these admissions had no prior outpatient visits (virtual or in-person) within 7 days of admission (Fig 1). Furthermore, 14% of patients admitted to ED had at least one virtual visit within the 7 days prior to ED admission, occurring on average 2.25 (SD = 2.31) days before the ED admission. Among those who had a virtual or an in-person visit within 7 days of ED admission, 34% had a virtual visit on the same day as the ED admission. (Table 1) In comparison, between July 1, 2019 and September 30, 2019, there were 1,113,230 ED admissions and 75% of admissions had no prior outpatient visits (virtual or in-person) within 7 days of admission and only 0.7% of admissions had a virtual visit 7 days prior to ED admission. (Table 1) Therefore, total ED admissions did not change significantly between 2019 and 2021, however, the availability of virtual visits was much higher in 2021 and as a result a larger percentage of

PLOS ONE
Virtual care use prior to emergency department admissions during a stable COVID-19 period in Ontario, Canada Table 1 Table 1 lists sociodemographic characteristics of all patients with an ED admission in 2021 and 2019 (Table 1 and also see S2 Appendix for some groups not shown in the manuscript)) across groups of patients with varying use of outpatient care prior to admission. When comparing the periods in 2021 vs. 2019, there were few differences in age, sex, region, rurality of residence, and neighborhood income between patients with no prior visits and patients with only in-person visits prior to their ED admission (<1% changes across). There were shifts in the characteristics of patients who had

Prior healthcare utilization
The average number of outpatient visits in the year prior to their ED admission across all patients admitted to ED in 2021 and in 2019 was similar (approximately 12 visits, Table 2 and also see S2 Appendix for some groups not shown in the manuscript). Patients with virtual visits prior to ED admission also had similar prior outpatient use in 2021 and 2019, but had more total outpatient visits relative to the entire ED user population (19 visits  When examining all patients with ED admissions, the average number of outpatient visits per patient in the 7 days prior to admission did not change (Mean = 1.3, SD = 0.7 in both 2021 and 2019). Patients with prior virtual visits were more likely to be hospitalized after their ED admission (13%) than patients with no prior outpatient care (7.7%), but patients with in-person visits prior to ED admissions were even more likely to be hospitalized (14.5%) (Fig 2).

PLOS ONE
Virtual care use prior to emergency department admissions during a stable COVID-19 period in Ontario, Canada infection (6%) and open wound of finger(s) (5%) across all ED admission. These reasons were consistent across patient groups with little variation on volumes of these admissions ( Table 3). The most common reasons for the last virtual visit prior to ED admission were "other illdefined conditions" (12%), gastrointestinal issues (12%), anxiety (11%), chest pain (8%) and leg cramps (7%). The same reasons were seen in the in-person visits only group with the exception that gastrointestinal and anxiety issues were the top two reasons for an in-person visit prior to ED admission (13%) ahead of "other ill-defined conditions" (Table 4).

Physician characteristics
There were no differences in physician characteristics (age, sex, years in practice, patient volume, and region of practice) between those who provided a virtual visit right before admission versus those who provided either a virtual or in-person visit in July 1, 2021-September 30, 2021. (Table 5)

PLOS ONE
Virtual care use prior to emergency department admissions during a stable COVID-19 period in Ontario, Canada

Interpretation
We found no rise in ED admission volumes and pre-admission outpatient care in the summer of 2021 relative to 2019. For patients with outpatient care in the 7 days prior to admission (25% of all patients), the modality of outpatient care shifted from patients having almost exclusively in-person visits in 2019 to patients having either virtual visits only or in-person care only in the week prior to their ED admission. Very few patients (3%) had a mix of both virtual and in-person care in the 7 days before their admission. Patients who had virtual visits prior to ED admission had more outpatient visits, but similar ED and hospitalization rates in the year prior to admission. They were also more likely to be hospitalized after their ED admission than patients with no prior visits. The most common reasons for ED admission were similar across groups of patients with varying levels of virtual care use prior to ED admission. Early in the pandemic, there were numerous reports of a decline in ED admissions across the globe [12][13][14]. However, stable pandemic periods, such as the one we report on in this study, have shown a return to regular ED use [14][15][16]. Consistent with these findings, ED admission volumes in Ontario remained nearly identical to those during a matched pre-pandemic period and the use of outpatient services prior to ED admissions did not increase. Therefore, the rise in use of virtual outpatient care in Ontario (about 50% of all outpatient care [17]) did not appear to lead to increased use of the ED. Furthermore, patients were admitted to the ED for similar reasons in both 2021 and 2019 with no evident shift in the top causes of ED admissions. Of note is that a greater proportion of those admitted to the ED were hospitalised in 2021 (9.14%) compared to 2019 (8.14%), which may suggest patients were more severely ill on ED admission. To our knowledge, this is the first study to look at the relationship between virtual visits use and ED use during the pandemic.
Patients who went to the ED with prior virtual visits were more likely to be hospitalized than those without any prior outpatient care (despite similar ED and hospital utilization in the year prior). This suggests that virtual visits were sufficient for physicians to advise patients to go to the emergency and an in-person outpatient visit was not required (only 3% of patients had a mix of virtual and in-person visits in the week prior to ED admission). Our finding is supported by the success of numerous emergency department clinics that introduced virtual visits during the pandemic [18][19][20].
Virtual visits were more prevalent in urban and higher income regions. Patients who went into the ED with a prior virtual visit in the week leading to their admission generally seemed to have better access to outpatient care, as evidenced by a higher number of outpatient visits in the year before their ED admission. While this may suggest greater severity of disease, there were no differences in the number of hospitalizations or ED admissions in the year prior to admission suggesting that the difference may lie in better access to outpatient care. This supports a growing concern that the shift towards virtual visits may be further limiting

PLOS ONE
Virtual care use prior to emergency department admissions during a stable COVID-19 period in Ontario, Canada marginalized patient populations' access to care [21]. This reinforces the notion that health equity considerations should always be at the forefront of the implementation of virtual care programs [22,23].
As virtual visits continue to be an integral part of medicine, it is reassuring to see that despite its high use during the COVID-19 pandemic, increases in ED use were not observed. For this study we examined patients who were visiting the ED, but what remains to be examined is how virtual visits affect downstream use of not only the ED, but also other types of outpatient care and diagnostics. The balance of virtual and in-person care may shift over time, so it is important to examine the relationship between virtual outpatient use and ED use and explore how these relationships change as practices build virtual visits into their workflows and better understand its value. Virtual visits may have differential impact on various ED admission diagnoses, which can be the subject of future work. It is noteworthy that despite the ease of virtual visits, similarly large proportions of patients go straight to the ED without seeking outpatient care before and during COVID, likely due to poor virtual visit access. This supports the idea that we need to make the type of care available in ED or holistic care more accessible overall so that more low acuity care shifts outside the emergency department [24].

Limitations
Limitations of this study include the use of administrative databases which lack the clinical granularity to assess details such as the appropriateness of the visit(s). Second, the recent temporary COVID-19 virtual billing codes do not distinguish between telephone and video, and therefore we were unable to make comparisons of the various modalities of virtual visits. Lastly, the findings in this report are descriptive only and span a short period of three months, and therefore results are only preliminary and may not be generalizable in different contexts.

Conclusions
Despite concerns that access to virtual visits may lead to a rise in ED admissions or a greater use of outpatient services prior to ED admissions (in the form of patients having both virtual and in-person visits), the net amount of ED admissions and outpatient care prior to admission remained the same over a period of the COVID-19 pandemic when cases were relatively stable. Virtual visits seem to be able to appropriately triage patients to the ED and may even prove beneficial for diverting patients away from the ED when an ED admission is not appropriate.

PLOS ONE
Virtual care use prior to emergency department admissions during a stable COVID-19 period in Ontario, Canada